Information processing apparatus and information processing method

ABSTRACT

An information processing apparatus that analyzes video data obtained from an image capture apparatus to detect an article that has been pre-registered is disclosed. The apparatus, in a case where an article that is determined to be a property of an article holder that has been pre-registered is detected, searches for an advertisement viewer located within a range of a predetermined distance from the article holder from among advertisement viewers that have registered the article as an article they are interested in. The apparatus then informs an advertisement viewer found by the search of location information of the article.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of U.S. patent application Ser. No.17/231,730, filed Apr. 15, 2021, which claims the benefit of JapanesePatent Application No. 2020-075606, filed Apr. 21, 2020, both of whichare hereby incorporated by reference herein in their entirety.

BACKGROUND Field of the Disclosure

The present disclosure relates to an information processing apparatusand an information processing method and particularly relates totechnology for providing information about an article.

Description of the Related Art

Typically, when collecting information about a product you areinterested in, you look at advertisement images and videos provided bycompanies on television, magazines, the Internet; go to a retail shop ora showroom and look and touch the actual product; and the like.

However, advertisement images and videos provided by companies areusually designed and made with the intent of drawing the attention ofviewers. Thus, it is difficult to determine the size, color, quality,and the like of the actual product. Also, the desired product is notalways at the retail shops and showrooms nearby. Furthermore, theproduct may be in the store, but where it is displayed may be hard todetermine.

In Japanese Patent Laid-Open No. 2010-9566, a system is described whichdetects when a customer who is strolling through the store is near anarticle registered before coming to the store, controls a notificationunit disposed on a display shelf, and informs the customer via audio,light, or the like.

With the system described in Japanese Patent Laid-Open No. 2010-9566,the time and effort of looking in the store for the desired product isreduced for the customer. However, the system is unable to provideinformation about products not registered beforehand and products notdisplayed in the store. Also, the system does not provide information tocustomers not in the store. Furthermore, regarding products such asdurable goods, the system is unable to provide information aboutproducts actually used in terms of the change of color over time and thelike.

SUMMARY

Embodiments of the present disclosure reduce one or more of the issuesin the related art. Specifically, embodiments of the present disclosureprovide an information processing apparatus and an informationprocessing method that assists a user in collecting information relatingto an article.

According to some embodiments of the present disclosure, there isprovided an information processing apparatus comprising one or moreprocessors that execute a program stored in a memory and function as: adetection unit configured to analyze video data obtained from an imagecapture apparatus to detect an article that has been pre-registered; asearch unit configured to, in a case where an article that is determinedto be a property of an article holder that has been pre-registered isdetected, search for an advertisement viewer located within a range of apredetermined distance from the article holder from among advertisementviewers that have registered the article as an article they areinterested in; and an informing unit configured to inform anadvertisement viewer found by the search unit of location information ofthe article.

According to some embodiments of the present disclosure, there isprovided an information processing method executed by an informationprocessing apparatus, comprising: analyzing video data obtained from animage capture apparatus to detect an article that has beenpre-registered; in a case where an article that is determined to be aproperty of an article holder that has been pre-registered is detected,searching for an advertisement viewer located within a range of apredetermined distance from the article holder from among advertisementviewers that have registered the article as an article they areinterested in; and informing an advertisement viewer found in thesearching of location information of the article.

According to some embodiments of the present disclosure, there isprovided a non-transitory computer-readable medium storing a program forcausing a computer to function as an information processing apparatuscomprising: a detection unit configured to analyze video data obtainedfrom an image capture apparatus to detect an article that has beenpre-registered; a search unit configured to, in a case where an articlethat is determined to be a property of an article holder that has beenpre-registered is detected, search for an advertisement viewer locatedwithin a range of a predetermined distance from the article holder fromamong advertisement viewers that have registered the article as anarticle they are interested in; and an informing unit configured toinform an advertisement viewer found by the search unit of locationinformation of the article.

Further features of the present disclosure will become apparent from thefollowing description of exemplary embodiments (with reference to theattached drawings).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram illustrating an overview of anadvertisement system according to an embodiment.

FIG. 2 is a block diagram illustrating an example of the functionalconfiguration of apparatuses forming an advertisement system accordingto an embodiment.

FIGS. 3A to 3C are flowcharts relating to an advertisement methodaccording to an embodiment.

FIG. 4 is a diagram illustrating an example of a display on aninformation terminal according to an embodiment.

FIG. 5 is a block diagram illustrating an example of the functionalconfiguration of an information terminal of an article holder and aninformation processing apparatus according to an embodiment.

FIG. 6 is a flowchart relating to registration processing for an articleholder according to an embodiment.

FIG. 7 is a diagram relating to registration processing for an articleholder according to an embodiment.

FIG. 8 is a diagram relating to registration processing for an articleholder according to an embodiment.

FIG. 9 is a diagram relating to registration processing for an articleholder according to an embodiment.

FIG. 10 is a diagram relating to registration processing for an articleholder according to an embodiment.

FIG. 11 is a flowchart relating to consideration assignment processingfor an article holder according to an embodiment.

FIG. 12 is a diagram relating to consideration assignment processing foran article holder according to an embodiment.

FIG. 13 is a flowchart relating to registration processing for anadvertisement viewer according to an embodiment.

FIG. 14 is a diagram relating to registration processing for anadvertisement viewer according to an embodiment.

FIG. 15 is a diagram relating to registration processing for anadvertisement viewer according to an embodiment.

FIG. 16 is a block diagram illustrating an example of the functionalconfiguration of an information terminal of an advertisement requesterand an information processing apparatus according to an embodiment.

FIG. 17 is a flowchart relating to registration processing for anadvertisement requester according to an embodiment.

FIGS. 18A to 18B are diagrams relating to registration processing for anadvertisement requester according to an embodiment.

FIG. 19 is a flowchart relating to advertisement rate invoice processingaccording to an embodiment.

FIGS. 20A and 20B are diagrams relating to advertisement rate invoiceprocessing according to an embodiment.

FIGS. 21A and 21B are schematic diagrams of the inside of a vehicleinstalled with an advertisement system according to a second embodiment.

FIG. 22 is a block diagram illustrating an example of the functionalconfiguration of apparatuses forming an advertisement system accordingto the second embodiment.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, embodiments will be described in detail with reference tothe attached drawings. Note, the following embodiments are not intendedto limit the scope of the disclosure. Multiple features are described inthe embodiments, but limitation is not made in this disclosure thatrequires all such features, and multiple such features may be combinedas appropriate. Furthermore, in the attached drawings, the samereference numerals are given to the same or similar configurations, andredundant description thereof is omitted.

First Embodiment

FIG. 1 is a schematic diagram illustrating an overview of anadvertisement system according to an embodiment of the presentdisclosure. An advertisement company 100 is a company or public officethat is the operator of the advertisement system. An article holder 108is an owner of an advertised target article 106 and may be an individualor group. An advertisement viewer 107 is an advertisement target and isa normal user (consumer). An advertisement requester 109 is anindividual or a group that advertises the target article 106 to theadvertisement viewer 107 using the advertisement system and is typicallythe maker or retail shop of the target article 106.

Overview of Advertisement System

The article holder 108, via a registration procedure 114 for theadvertisement company 100, grants permission to the advertisementcompany 100 to provide their location information to the advertisementviewer 107 and/or grants permission for an image of their face to beused in subject recognition processing. The article holder 108 receivesa consideration 117 from the advertisement company 100 for grantingpermission to provide the location information. Note that the articleholder 108 may additionally or instead of a consideration for grantingpermission receive the consideration 117 for actually providing thelocation information.

The advertisement viewer 107 who wishes to use the advertised service(view the advertisement) provided by the advertisement company 100enters into an agreement to view the advertisement with theadvertisement company 100 via a registration procedure 113. Theadvertisement viewer 107, via the registration procedure 113, canregister the type of articles, specific product names, and the like ofarticles they are interested in or are thinking of buying. Also, theadvertisement viewer 107 may grant permission to the advertisementcompany 100 to the user search history information of the search servicethey used on their computer or portable terminal.

Note that in a case where the viewer-targeted advertisement serviceprovided by the advertisement company 100 is a paid service, anagreement may be included in the agreement between the advertisementviewer 107 and the advertisement company 100 in the registrationprocedure 113 relating to payment of a consideration such as a servicefee.

The advertisement requester 109 who wishes to use the advertised service(provide the advertisement) provided by the advertisement company 100enters into an agreement to provide the advertisement with theadvertisement company 100 via a registration procedure 115. Theadvertisement requester 109, via the registration procedure 115,registers the article for advertisement and enters into an agreementrelating to payment of the advertisement rate. When the article isregistered, a photo of the article and other information thatfacilitates the recognition of the article from a captured image can beregistered. Note that the advertisement rate may be calculated on thebasis of the number of times the advertisement is run or may be a fixedamount.

To use in the advertisement, the advertisement company 100 obtains avideo taken in real time by an image capture apparatus 101 with a knownplacement position, such as a video camera placed inside a facility oroutside. The image capture apparatus 101 may be managed by theadvertisement company 100 or may be managed by another party. In a caseof the latter, an agreement relating to provision of the video may beconcluded between the party managing the image capture apparatus 101 andthe advertisement company 100, for example.

Next, a method of providing an advertisement to the advertisement viewer107 using an advertisement system will be described. In an informationprocessing apparatus 102 (for the sake of convenience, this is listed asa single component, but in reality, multiple information processingapparatuses are used) managed by the advertisement company 100 isprovided with a video signal from the image capture apparatus 101 inreal time. Also the location information of the advertisement viewer 107and the article holder 108 are provided by a mobile phone company atpredetermined time intervals, for example.

The information processing apparatus 102 analyzes the video providedfrom the image capture apparatus 101 and detects the target article 106registered by the advertisement requester 109. Also, the informationprocessing apparatus 102 determines whether the article holder 108 isshown in the video on the basis of the relationship between the locationinformation of the image capture apparatus 101 and the locationinformation of the article holder 108, the detection result of a faceregion of the article holder 108, and the like. Note that in addition tothe location information of the image capture apparatus 101, informationrelating to the image capture direction and angle of view may be used inthis determination.

The information processing apparatus 102 identifies the article holder108 holding the target article 106 according to a predetermineddetermination condition. Note that the information processing apparatus102 transmit an SMS or the like including a description of the targetarticle 106 to the information terminal 104 or the like of the articleholder 108 considered to be holding the target article 106, and thearticle holder 108 may confirm that they are actually holding the targetarticle 106. Note that in FIG. 1 , the article holder 108 is illustratedmoving while carrying the target article 106. However, the articleholder 108 may not be carrying the target article 106. However, in acase where the location information of the article holder 108 isinformed as location information of the target article 106, it ispresumed that the target article 106 and the article holder 108 are notfar apart in terms of distance.

Also, the information processing apparatus 102 determines whether or notthe advertisement viewer 107 that has registered the target article 106held by the identified article holder 108 as an article they areinterested in is located close to the article holder 108. For example,the information processing apparatus 102 is capable of determining thetwo as being located close to one another in a case where the distancecalculated from the location information of the article holder 108 andthe advertisement viewer 107 is equal to or less than a threshold.

In a case where the information processing apparatus 102 determines thatthe advertisement viewer 107 that has registered the article held by thearticle holder 108 as an article they are interested in is located closeto the article holder 108, the information processing apparatus 102informs the advertisement viewer 107 of the location of the articleholder 108 (the target article 106). For example, the informationprocessing apparatus 102 is capable of informing of the location of thetarget article 106 by transmitting a message to an information terminal103 of the advertisement viewer 107, for example.

FIG. 2 is a block diagram illustrating an example of the functionalconfiguration of the image capture apparatus 101, the informationprocessing apparatus 102, and the information terminal 103 owned by theadvertisement viewer 107.

The image capture apparatus 101 is a fixed point camera, for example,and includes an image capture circuit 201, a calculation circuit 202, atransmission circuit 203, and a storage device 204. The image capturecircuit 201 includes a lens and an image sensor. The lens forms anoptical image of a subject on an imaging surface of the image sensor.The lens angle of view and the image capture direction may be fixed orchangeable. The image sensor is a CMOS image sensor, for example, and isprovided with a plurality of pixels that include a photoelectricconverter circuit. The image sensor converts the subject optical imagevia the pixels into an electrical signal group (analog image signal).Herein, the image sensor includes an A/D converter and outputs imagedata which is the A/D converted analog image signal.

The calculation circuit 202 is a CPU, for example, that loads a programstored on a ROM of the storage device 204 on a RAM of the storage device204 and executes the program. By the program being executed, thecalculation circuit 202 implements various operations of the imagecapture apparatus 101, such as controlling the circuits inside the imagecapture apparatus 101, communicating with an external device, and thelike. Note that the calculation circuit 202 may include dedicatedhardware for processing the image data for use in image processing. Thecalculation circuit 202 reads out image data from the image captureapparatus at a predetermined frame rate and generates moving image data(video data) using known image processing (for example, developmentprocessing, encoding processing, and the like). Also, the calculationcircuit 202 may continuously execute automatic focus adjustment (AF) andautomatic exposure control (AE) of the lens on the basis of the videodata. Furthermore, the calculation circuit 202 is capable of executinganalysis processing as a part of the image processing. The calculationcircuit 202 is capable of changing the setting value of the imagecapture apparatus 101, the lens angle of view, the focusing distance,and the image capture direction according to a command received from anexternal device.

The transmission circuit 203 is a communication interface between anexternal device and the image capture apparatus 101 and is compliantwith at least one wired and/or wireless communication protocol. Notethat in FIGS. 1 and 2 , the image capture apparatus 101 and theinformation processing apparatus 102 are illustrated as communicatingdirectly. However, they may communicate via a network. The calculationcircuit 202 streams the video data to the information processingapparatus 102 via the transmission circuit 203. Note that video data ofthe image capture apparatus 101 managed by a party other than theadvertisement company 100 may be provided to the information processingapparatus 102 via the administrator.

The storage device 204 includes the ROM, the RAM, a hard disk drive, andthe like. The ROM stores programs executed by the calculation circuit202, various setting values of the image capture apparatus 101, and thelike. The RAM is used when the calculation circuit 202 executesprograms, as buffer memory for the video data, and the like. The harddisk drive (or another large capacity storage device) stores the mostrecent predetermined amount of time of video data, for example.

The information processing apparatus 102 is a computer device andincludes a transmission circuit 205, a calculation circuit 206, and astorage device 207. In FIG. 2 , one apparatus is illustrated. However,in reality, the information processing apparatus 102 may be an assemblyof a plurality of computer devices. The transmission circuit 205 is acommunication interface between an external device and the informationprocessing apparatus 102 and is compliant with at least one wired and/orwireless communication protocol. In FIG. 2 , the single transmissioncircuit 205 is illustrated as communicating to the image captureapparatus 101 and the information terminal 103. However, thetransmission circuit 205 may differ depending on the communicationmethod. Also, the communication between the image capture apparatus 101and the information terminal 103 may be executed via a network.

The calculation circuit 206 is a CPU, for example, that loads a programstored on a ROM of the storage device 207 on a RAM of the storage device207 and executes the program. By the program being executed, thecalculation circuit 206 causes the information processing apparatus 102to function as an advertisement providing apparatus, by controlling thecircuits inside the information processing apparatus 102, communicatingwith an external device, and the like.

The calculation circuit 206 executes registration processing for usersof the advertisement providing system including the article holder 108,the advertisement viewer 107, and the advertisement requester 109 andprotects and manages information relating to the users and articlesobtained in the registration processing and user location information.Also, the calculation circuit 206 analyzes the video data obtained fromthe image capture apparatus 101 and determines whether or not theidentified advertisement viewer 107 is located near the article holder108. Furthermore, in a case where it is determined that theadvertisement viewer 107 is located near the article holder 108corresponding to a condition, the calculation circuit 206 transmitsinformation of the article holder 108 to the advertisement viewer 107.

The storage device 207 includes the ROM, the RAM, a hard disk drive, andthe like. The ROM stores programs executed by the calculation circuit206, various setting values of the information processing apparatus 102,and the like. The RAM is used when the calculation circuit 206 executesprograms, as buffer memory for various data, and the like. The hard diskdrive (or a large capacity storage device) stores registrationinformation of the users (the advertisement viewer 107, the articleholder 108, and the advertisement requester 109). Also, informationrelating to the target article 106 advertisement, the current locationinformation of the advertisement viewer 107 and the article holder 108,and information of the image capture apparatus 101 are also stored inthe hard disk drive, and the calculation circuit 206 updates these asnecessary.

The information terminal 103 is a small computer device, such as a smartphone, a tablet computer, a laptop computer, or the like. Theinformation terminal 103 includes a transmission circuit 208, acalculation circuit 209, a display device 210, a positioning device 211,a storage device 212, and an input circuit 213.

The calculation circuit 209 is a CPU, for example, that loads a programstored on a ROM of the storage device 212 on a RAM of the storage device212 and executes the program. By the program being executed, thecalculation circuit 209 implements various operations of the informationterminal 103, such as controlling the circuits inside the informationterminal 103, communicating with an external device, and the like.

The transmission circuit 208 is a communication interface between anexternal device and the information terminal 103 and is compliant withat least one wired and/or wireless communication protocol. Typically,the transmission circuit 208 includes a communication interfacecompliant with a mobile communications protocol, such as 3G or LTE, anda communication interface compliant with a general-purpose wirelesscommunication protocol, such as wireless LAN or Bluetooth (registeredtrademark). Note that in FIGS. 1 and 2 , the information terminal 103and the information processing apparatus 102 are illustrated ascommunicating directly. However, they may communicate via a network. Thecalculation circuit 209 receives a notification relating to theadvertisement from the information processing apparatus 102 via thetransmission circuit 208.

The storage device 212 includes the ROM, the RAM, an SSD, and the like.The ROM stores programs executed by the calculation circuit 209, varioussetting values of the information terminal 103, and the like. The RAM isused when the calculation circuit 209 executes programs, as buffermemory for various data, and the like. The SSD (or a large capacitystorage device) stores information of the advertisement viewer 107, userdata, applications, and the like.

The display device 210 is a liquid crystal display, for example, anddisplays various displays according to a program executed by thecalculation circuit 209. The display device 210 may be a touch display.

The positioning device 211 includes a Global Positioning System (GPS)receiver, for example, and calculates the location informationindicating the geographic location of the information terminal 103 onthe basis of the reception signal. Note that because the informationterminal 103 is carried by the advertisement viewer 107, the informationprocessing apparatus 102 treats the location information of theinformation terminal 103 as the location information of theadvertisement viewer 107. Note that the positioning device 211 may uselocation information of an access point of a wireless LAN, a GPS signalreceived by a base station, or the like as auxiliary information tomeasure the location of the information terminal 103.

The input circuit 213 is an input device, such as a touch panel or akeyboard, operated by the advertisement viewer 107.

Overview of Advertisement Providing Operation

FIGS. 3A to 3C are flowcharts relating to the operations of theinformation processing apparatus 102. As illustrated in FIG. 3A, theinformation processing apparatus 102 starts obtaining video in step S301and continuously executes recognition processing in step S302 andtracking processing in step S303. In step S301, the calculation circuit206 starts obtaining video data from the image capture apparatus 101 viathe transmission circuit 205. The calculation circuit 206 temporarilystores the obtained video data in the storage device 207. Note thatunique information of the image capture apparatus 101 is included in thevideo data output by the image capture apparatus 101, and theinformation processing apparatus 102 is capable of identifying the imagecapture apparatus 101 that captured the video data and the locationthereof on the basis of the unique information.

In step S302, the calculation circuit 206 (a detection unit) appliesrecognition processing of the target article 106 to the obtained videodata. From the result of the recognition processing and the locationinformation and registration information of the article holder 108 andthe advertisement viewer 107, the calculation circuit 206 identifies thevideo data for tracking processing of the article holder 108 and theadvertisement viewer 107 satisfying a specific condition.

In step S303, the calculation circuit 206 executes the trackingprocessing of the article holder 108 and the advertisement viewer 107 onthe video data determined as the target for tracking processing in stepS302. Note that the calculation circuit 206 tracks changes in thedetection state of the target article 106 by continuously executing theprocessing of step S302 and step S303 on all of the video data.

The recognition processing of step S302 will now be described in detailusing the flowchart of FIG. 3B. In step S306, the calculation circuit206 applies recognition processing on each piece of video data (videostream) obtained in step S301 and determines whether or not theregistered target article 106 is shown. The calculation circuit 206 iscapable of using the information (for example, an image of the targetarticle 106) of the target article 106 that the advertisement requester109 provided when registering the target article 106, for example, andapplying a known recognition method to the video data. Examples of suchmethods include template matching using the image of the target article106 as a template, a method using a neural network trained using theimage of the target article 106, and the like.

In step S307, as the recognition processing result, the calculationcircuit 206 determines for each piece of video data whether or not oneor more types of the target article 106 are shown. For video data forwhich it is determined that the target article 106 is not shown, thecalculation circuit 206 determines that the article holder 108 and theadvertisement viewer 107 targeted for tracking processing are notpresent, and the following processing is skipped. For video data forwhich it is determined that the target article 106 is shown, thecalculation circuit 206 executes the processing of step S308.

In step S308, the calculation circuit 206 (determination unit)determines whether or not the detected target article is the property ofthe registered article holder 108. The calculation circuit 206 iscapable of determining this on the basis of image recognition processingexecuted on the video data and location information. For example, thecalculation circuit 206 uses face information provided at the time ofregistration by the article holder 108 that owns the detected targetarticle 106 to execute image recognition processing. Also, in a casewhere the face region of the article holder 108 is detected within adistance less than a threshold from the target article 106 in the image,the calculation circuit 206 determines that the target article 106 isthe property of the article holder 108. Alternatively, from the locationinformation of the image capture apparatus 101 that captured the videodata and the location information of the article holder 108 that ownsthe detected target article 106, the calculation circuit 206 determineswhether or not the article holder 108 with the distance to the imagecapture apparatus 101 less than a threshold exists. In a case where thecalculation circuit 206 determines that the article holder 108 that ownsthe detected target article 106 exists within a distance from the imagecapture apparatus 101 less than the threshold, the calculation circuit206 determines that the target article 106 is the property of thearticle holder 108. Note that these two methods may be used incombination or another method may be used.

Also, note that in a case where the location information is used, inaddition to the location information of the image capture apparatus 101,image capture direction information and lens focal length informationmay be used. In this case, instead of just the distance from the imagecapture apparatus 101, whether the article holder 108 is present in theimage capture direction (or within the image capture range) isconfirmed, thus increasing determination accuracy. Note that thecalculation circuit 206 may confirm the personal identity bytransmitting, to the portable terminal of the article holder 108determined to own the detected target article 106, a message forconfirming that the determination by the calculation circuit 206 iscorrect. Also, the detected target article 106 is treated as theproperty of the identified article holder 108 only when the personalidentity is confirmed by the calculation circuit 206.

Furthermore, the confirmation message may be a query as to whether ornot the article holder 108 will provide their location information. Byanswering the confirmation message with a no to providing locationinformation, the article holder 108 can temporarily refuse to providelocation information. The information processing apparatus 102 informsthe advertisement viewer 107 of location information only when an answerof yes is received for providing location information. Accordingly, thiscan prevent the location information from being provided when thearticle holder 108 does not wish to provide the location information,and the privacy of the article holder 108 can be protected.

Note that the information of the target article 106 and the articleholder 108 used in the image recognition processing in step S306 andstep S308 may be obtained from an external device instead of or inaddition to being obtained from the storage device 207 of theinformation processing apparatus 102.

For video data for which it is determined that the article holder 108that owns the target article 106 is not present, the calculation circuit206 determines that the target for tracking processing is not present,and the following processing is skipped. In a case where the calculationcircuit 206 determines that the article holder 108 determined to own thetarget article 106 is present, the processing of step S309 is executed.

In step S309, the calculation circuit 206 (searching unit) determineswhether or not any advertisement viewer 107 that registered the targetarticle 106 is present within the range of a predetermined distance fromthe article holder 108 being determined to own the target article 106.In this determination, the calculation circuit 206 is capable of usingthe location information of the article holder 108 determined to own thetarget article 106 and the location information of every advertisementviewer 107 that registered the target article 106. Note that thecalculation circuit 206 may determine that an advertisement viewer 107shown in the same image as the target article 106 to be present with thepredetermined distance from the article holder 108 determined to own thetarget article 106. The calculation circuit 206 is capable ofdetermining whether or not an advertisement viewer 107 is shown byapplying to the video data subject detection processing using the faceinformation obtained when the advertisement viewer 107 registered.

Note that an advertisement viewer 107 that is merely viewing the targetarticle 106 and has not registered the target article 106 may be treatedin a similar manner as an advertisement viewer 107 that has registeredthe target article 106. From the relationship between the orientation ofthe face of an advertisement viewer 107 detected by the subjectdetection processing and the location of the target article 106, thecalculation circuit 206 is capable of detecting the advertisement viewer107 considered to be viewing the target article 106.

In step S310, the calculation circuit 206 determines that the target oftracking processing is not present in a case where any advertisementviewer 107 that registered the target article 106 is not present withinthe range of the predetermined distance from the article holder 108determined to own the target article 106. On the other hand, thecalculation circuit 206 sets the article holder 108 and theadvertisement viewer 107 as targets for tracking processing in a casewhere an advertisement viewer 107 that registered the target article 106is determined to be present within the predetermined distance from thearticle holder 108 determined to own the target article 106. Note thatthe tracking targets, the article holder 108 and the advertisementviewer 107, are not limited to having a one-to-one correlation and mayhave a one-to-many correlation or a many to one correlation.

Via the processing described above, the calculation circuit 206determines whether or not there is a tracking target for each of thetarget articles 106 detected from the video data. The calculationcircuit 206 stores the information identifying the tracking targets, thearticle holder 108 and the advertisement viewer 107, in the RAM of thestorage device 207.

The tracking processing of step S303 will now be described in detailusing the flowchart of FIG. 3C. The information processing apparatus 102executes the tracking processing illustrated in FIG. 3C for eachcombination of the article holder 108 and the advertisement viewer 107set as a target for tracking processing in the recognition processing ofstep S302.

In step S313, the calculation circuit 206 determines the location of thetracking targets, the article holder 108 and the advertisement viewer107, on the basis of their location information. The locationinformation of the advertisement viewer 107 may be the locationinformation measured by the positioning device 211 obtained from theinformation terminal 103. Also, the location information of the articleholder 108 may be the location information of the information terminal104 of the article holder 108. A calculation circuit 206 is capable ofobtaining the location information of the information terminal 104directly from the information terminal 104 or via the mobile phonecompany.

Note that the location of the article holder 108 and the advertisementviewer 107 may be identified without using the location informationmeasured by the information terminals 104 and 103. For example, thelocation of the article holder 108 may be identified from the locationinformation of the image capture apparatus 101 capturing an image of thearticle holder 108, and the location of the advertisement viewer 107 maybe identified from the location information of the image captureapparatus 101 capturing an image of the advertisement viewer 107. Inthis case, by using the information relating to image capture directionand angle of view, the accuracy of the location can be improved. Also, acase where a plurality of the image capture apparatus 101 capture thesame article holder 108 (or advertisement viewer 107) at the same timeallows the accuracy of the location to be improved. The locationinformation obtained from the information terminals 103 and 104 or themobile phone company and the location information of the image captureapparatus 101 may be used together or switched between. For example, thelocation information of the image capture apparatus 101 may be used in acase where the location information cannot be obtained from a portableterminal or the mobile phone company.

Note that in a case where the distance obtained from the locationrelationship between the article holder 108 and the advertisement viewer107 is greater than the threshold consistently for a certain amount oftime (for example, the threshold used in step S309), the calculationcircuit 206 may remove the article holder 108 and the advertisementviewer 107 from being tracking targets. Note that at this point in time,the location information of the article holder 108 has already beenprovided to the advertisement viewer 107 and the information terminal103. Accordingly, the calculation circuit 206 may confirm with theadvertisement viewer 107 before ending provision of the locationinformation of the article holder 108 whether it is ok to end provision.The calculation circuit 206, for example, transmits a confirmationmessage to the information terminal 103 of the advertisement viewer 107and, in a case where permission is obtained, removes the article holder108 and the advertisement viewer 107 from being tracking targets. Thisends the provision of the location information of the article holder 108to the advertisement viewer 107. On the other hand, in a case where theresponse to the confirmation message to an instruction to continueprovision of the location information, the calculation circuit 206continues to provide the location information for a certain amount oftime until the distance between the advertisement viewer 107 and thearticle holder 108 exceeds the threshold.

In step S315, the calculation circuit 206 (informing unit) transmits thelocation information of the article holder 108 to the informationterminal 103 of the advertisement viewer 107 via the transmissioncircuit 205. The calculation circuit 209 of the information terminal 103informs the user (the advertisement viewer 107) that the locationinformation will be received. When the user performs an instruction todisplay the location information, the calculation circuit 209 using amap application installed in the information terminal 103, for example,to display the location of the article holder 108 on the display device210. Also, the calculation circuit 209 may use the navigation functionof the map application to display the route from the advertisementviewer 107 to the article holder 108 and provide navigation thereto.Note that the calculation circuit 209 may display the location of thearticle holder 108 and information (for example, an image extracted fromthe video data, an image prepared in advance, an article name, and thelike) about the target article 106 together. Also, the informationterminal 103 may include a gyro sensor and the like, and the calculationcircuit 209, in a case where the orientation of the terminal is able tobe detected, may inform of what direction the article holder 108 is ininstead of or in addition to displaying the location on the map.

In step S316, the calculation circuit 206 determines whether or not anend viewing instruction has been received from the information terminal103 of the advertisement viewer 107 via the transmission circuit 205. Ina case where an end viewing instruction has not been received, thecalculation circuit 206 repeatedly executes the processing again fromstep S313. Also, in a case where an end viewing instruction has beenreceived, the calculation circuit 206 removes the combination of thearticle holder 108 and the advertisement viewer 107 from being trackingtargets.

As described above, the calculation circuit 206 continuously executesstep S302 to detect new tracking targets while also continuouslyexecuting the tracking processing of step S303 on each tracking target.

FIG. 4 is a diagram illustrating an example of a display on theinformation terminal 103 that has received the location information (aninformation notification 112 illustrated in FIG. 1 ) of the articleholder 108 from the information processing apparatus 102 in step S315.In the example illustrated in FIG. 4 , by displaying on the displaydevice 210 a message 401 indicating the direction where the articleholder 108 is, the advertisement viewer 107 is informed of the locationof the article holder 108. Also, information about the target article106 is included in the message 401. As described above, instead of onlydisplaying the direction, the location of the article holder 108 on amap may be displayed, or the route from the advertisement viewer 107 tothe article holder 108 may be displayed and navigation thereto may beprovided.

Also, in the example illustrated in FIG. 4 , the message 401 as well asan end button 402 is displayed. When the end button 402 is operated(touched), the calculation circuit 209 transmits an end viewinginstruction to the information processing apparatus 102 and ends thedisplay of the location information of the article holder 108.

Note that instead of or in addition to informing of the locationinformation of the article holder 108 in step S315, a video of thearticle holder 108 may be provided to the information terminal 103 ofthe advertisement viewer 107. For example, in a case where the articleholder 108 is in a crowded area, it may be difficult to identify thearticle holder 108. By providing a video of the article holder 108, thearticle holder 108 is made easier to find and information about theactual use state of the target article 106 can be obtained from thevideo without needing to be near the article holder 108.

According to the advertisement system of the present embodiment, by theadvertisement viewer 107 registering an article they are interested inas the target article 106, the advertisement viewer 107 can be informedthat the actual target article 106 is nearby. Accordingly, the actualtarget article 106 can be confirmed by taking advantage of the fact thatthe target article 106 is nearby. This allows for information that isunable to be obtained from the state of the article when on display inan advertisement or in the store to be easily obtained. Also, in a casewhere the article holder 108 is carrying or using the target article106, trends relating to the age, gender, and the like of the users canbe known, allowing for an informed choice to be made when purchasing thetarget article 106.

The advertisement system of the present embodiment has been describedabove in terms of the operations of the information processing apparatus102 at the center of the system mainly from the perspective of providinginformation to the advertisement viewer 107. Other operations of theinformation processing apparatus 102 will be described below.

Processing of Registration and Consideration Payment for Article HolderFirst, the operations of the information processing apparatus 102relating the registration procedure 114 and the consideration 117 forthe article holder 108 illustrated in FIG. 1 will be described. FIG. 5is a block diagram illustrating the information terminal 104 of thearticle holder 108 and the information processing apparatus 102 and thefunctional configuration thereof relating to the registration procedure114 and the consideration 117.

The information terminal 104 is a computer device owned by the articleholder 108. In this example, the information terminal 104 is a portablecomputer device, such as a smart phone, a tablet terminal, or a notebookcomputer. However, the information terminal 104 may be a stationarycomputer device such as a desktop computer.

The information terminal 104 and the information processing apparatus102 are capable of communicating in both directions via transmissioncircuits 501 and 205, respectively. Note that a network may existbetween the information terminal 104 and the information processingapparatus 102. The transmission circuits 501 and 205 communicateaccording to a wired or wireless communication protocol both arecompliant to. In a case where the information terminal 104 is a smartphone, for example, the information terminal 104 and the informationprocessing apparatus 102 communicate in both directions via a mobilecommunication network or directly via a wireless LAN or Bluetooth(registered trademark).

A calculation circuit 502 is a CPU, for example, that loads a programstored on a ROM of a storage device 505 on a RAM of the storage device505 and executes the program. By the program being executed, thecalculation circuit 502 makes the information terminal 104 executevarious operations, such as controlling the circuits inside theinformation terminal 104, communicating with an external device, and thelike. Included in the program stored on the ROM of the storage device505 may be various typical application programs installed on a computerdevice, such as a web browser, a camera application, and the like.

An input circuit 503 is an input device, such as a touch panel or akeyboard, operated by the article holder 108. A display device 504 maybe a liquid crystal display or an organic EL display, for example, andmay be a touch display.

The configuration of the information processing apparatus 102 is asdescribed in reference to FIG. 2 . The information of each articleholder 108 is stored in a non-volatile storage device such as the ROM orSSD of the storage device 207. Included in the information of thearticle holder 108 may be, but is not limited to, information (personalinformation, article information, password, and the like) input at thetime of registration, consideration information, and the like.

The calculation circuit 206 causes the information processing apparatus102 to function as a web server by executing a program, for example, andprovides information for displaying various input forms to the webbrowser that accesses the web server.

FIG. 6 is a flowchart relating to the operations of the informationprocessing apparatus 102 relating to registration (the registrationprocedure 114 of FIG. 1 ) of the article holder 108.

In step S602, the calculation circuit 206 replies to access to a website(URL) for article holder registration from the web browser applicationrun by the information terminal 104 and transmits data of a personalinformation input screen to the information terminal 104. Thecalculation circuit 502 uses the web browser application and causes thedisplay device 504 to display the personal information input screen.

FIG. 7 is an example of a personal information input screen. In thisexample, the entire display device 504 is used for the web browserapplication. Also, the display device 504 is a touch display and alsofunctions as the input circuit 503. The article holder 108 inputspersonal information in the input items by operating the display device504 via touch and ends input of personal information by touching aregistration button 701. When a touch operation on the registrationbutton 701 is confirmed, the calculation circuit 502 transmits the itemsinput on the screen to the information processing apparatus 102. Whenthe personal information is received, the calculation circuit 206generates an article holder record including a unique user ID, storesthe received personal information in a predetermined field in therecord, and stores this in the storage device 207. Note that though notillustrated in the example of FIG. 7 , face information of the articleholder 108 may be registered as personal information. In this case, dataof a photo of the face of the article holder 108 taken using a camera ofthe information terminal 104 or the like may be included in theregistration information. Also, a password for logging into the websitefor registrants may be included in the registration information.

In step S603, the calculation circuit 206 transmits data of theinformation input screen of the target article to the informationterminal 104. FIG. 8 is a diagram illustrating an example of an inputscreen of the target article 106. In this example, an input screen isillustrated for registering a target article by selecting the maker andname or model of the product from a pull-down menu. However, the targetarticle may be registered using another information that can be used toidentify the target article. For example, a method may be used in whicha photo of the target article is transmitted and the informationprocessing apparatus 102 recognizes the target article. When a touchoperation on a registration button 801 is confirmed, the calculationcircuit 502 transmits the contents selected from the pull-down menu tothe information processing apparatus 102. When the target articleinformation is received, the calculation circuit 206 registers theinformation in the article holder record in the storage device 207.

In step S604, the calculation circuit 206 transmits, to the informationterminal 104, data of an agreement input screen for granting permissionto the advertisement company and the advertisement viewer to use thelocation information of the information terminal 104. FIG. 9 is adiagram illustrating an example of an agreement input screen forlocation information. The agreement input screen includes wording 901explaining the contents of the permission grant, an agree (grantpermission) button 902, and a do not agree (do not grant permission)button 903. When a touch operation on the agree button 902 is confirmed,the calculation circuit 502 informs the information processing apparatus102 that agreement has been obtained. The calculation circuit 206replies to the notification of agreement and formally registers thearticle holder 108. On the other hand, when a touch operation on the donot agree button 903 is confirmed, the calculation circuit 502 informsthe information processing apparatus 102 that agreement has not beenobtained. The calculation circuit 206 replies to the notification ofnon-agreement and discards the registration of the article holder 108.In this case, step S605 is not executed.

In step S605, the calculation circuit 206 transmits, to the informationterminal 104, data of a notification setting screen for setting whetherto authorize notifications of the location information of theinformation terminal 104. FIG. 10 is a diagram illustrating an exampleof a notification setting screen for location information. Thenotification setting screen includes wording 1001 for explaining thesettings, an enable button 1002, and a disable button 1003. When a touchoperation on the enable button 1002 is confirmed, the calculationcircuit 502 instructs the information processing apparatus 102 thatnotifications have been turned on or authorized. On the other hand, whena touch operation on the disable button 1003 is confirmed, thecalculation circuit 502 instructs the information processing apparatus102 that notifications have been turned off or unauthorized. Thecalculation circuit 206 registers the information indicatingnotifications being on or off in the record of the article holder 108.The calculation circuit 206 sends notifications of the locationinformation to the advertisement viewer 107 only for the article holders108 (the information terminals 104) that have notifications set to on.

The information of the article holder 108 registered via the screensdescribed above may be changed at any time by the article holder 108 byusing the information terminal 104 to log in to the website of theadvertisement service provided by the information processing apparatus102. Also, as described above, before the advertisement viewer 107 isactually informed of the location information of the article holder 108(the information terminal 104), data of the notification setting screenmay be transmitted from the information processing apparatus 102 to theinformation terminal 104. This allows for the location information to beinformed only in cases where consent (notifications turned on) of thearticle holder 108 has been obtained.

Note that, in this example, the information terminal 104 uses a browserapplication and repeats the reception of the input screen and thetransmission of registration information while communicating with theinformation processing apparatus 102. However, a dedicated applicationincluding the input screens may be installed in the information terminal104, and all of the information may be transmitted as registrationinformation from the information terminal 104 to the informationprocessing apparatus 102 after all the information has finished beinginput.

FIG. 11 is a flowchart relating to the operations of the informationprocessing apparatus 102 relating to providing consideration (theconsideration 117 of FIG. 1 ) to the article holder 108. This processingmay be executed at specific periods, for example once every month.

In step S1101, the calculation circuit 206 identifies the article holder108. The calculation circuit 206 may sequentially identify the articleholder records stored in the storage device 207 in order of user ID, forexample.

In step S1102, the calculation circuit 206 searches for the number oftimes the advertisement viewer 107 has been informed of the locationinformation of an identified article holder. Note that the number oftimes may be the number of times in step S315 the location informationof an identified article holder 108 has been informed or may be thetotal number of advertisement viewers 107 that have been informed of thelocation information. In both cases, the calculation circuit 206measures and stores the number of times for each article holder 108 whentracking processing is executed.

In step S1103, the calculation circuit 206 determines the considerationbased on the number of times. In this example, the considerationcorresponds to points usable at a specific company, and the relationshipbetween the number of times and the points is determined in advance.

In step S1104, the calculation circuit 206 transmits a point assignmentnotification to the information terminal 104 of the article holder 108identified in step S1011. When the point assignment notification isreceived, the calculation circuit 502 of the information terminal 104displays the point assignment notification screen on the display device504 and informs the article holder 108. FIG. 12 is a diagramillustrating an example of the point assignment notification screen. Thepoint assignment notification screen include wording 1201 for explainingthe assignment contents and a close button 1202. When a touch operationon the close button 1202 is confirmed, the calculation circuit 502closes the point assignment notification screen.

Next, the operations of the information processing apparatus 102relating the registration procedure 113 of the advertisement viewer 107illustrated in FIG. 1 will be described. Note that the functionalconfiguration of the information terminal 103 of the advertisementviewer 107 and the information processing apparatus 102 is as describedin reference to FIG. 2 . As described above, the calculation circuit 206causes the information processing apparatus 102 to function as a webserver by executing a program, for example, and provides information fordisplaying various input forms to the web browser that accesses the webserver.

Advertisement Viewer Registration Processing

FIG. 13 is a flowchart relating to the operations of the informationprocessing apparatus 102 relating to registration (the registrationprocedure 113 of FIG. 1 ) of the advertisement viewer 107.

In step S1403, the calculation circuit 206 replies to access to awebsite (URL) for advertisement viewer registration from the web browserapplication run by the information terminal 103 and transmits data of apersonal information input screen to the information terminal 103. Thecalculation circuit 209 uses the web browser application and causes thedisplay device 210 to display the personal information input screen. Thepersonal information input screen may be the same as that described withreference to FIG. 7 .

When the personal information is received from the information terminal103, the calculation circuit 206 of the information processing apparatus102 generates an advertisement viewer record including a unique user ID,stores the received personal information in a predetermined field in therecord, and stores this in the storage device 207. Note that faceinformation may be registered as personal information for theadvertisement viewer in a similar manner as for the article holder. Inthis case, data of a photo of the face of the advertisement viewer 107taken using a camera of the information terminal 103 or the like may beincluded in the registration information. Also, a password for logginginto the website for registrants may be included in the registrationinformation.

In step S1404, the calculation circuit 206 transmits data of theinformation input screen of the target article to the informationterminal 103. FIG. 14 is a diagram illustrating an example of an inputscreen of the target article 106. Basically, this screen may be the samescreen used by the article holder to register the target article 106. Asillustrated in FIG. 14 , an input screen is illustrated for registeringa target article by selecting the maker and name or model of the productfrom a pull-down menu 1501. However, the target article may beregistered using another information that can be used to identify thetarget article. For example, a method may be used in which a photo ofthe target article is transmitted and the information processingapparatus 102 recognizes the target article. When a touch operation on aregistration button 1502 is confirmed, the calculation circuit 209transmits the contents selected from the pull-down menu to theinformation processing apparatus 102. When the target articleinformation is received, the calculation circuit 206 registers theinformation in the advertisement viewer record in the storage device207.

In step S1405, the calculation circuit 206 transmits, to the informationterminal 103, data of a notification setting screen for setting whetherto authorize notifications of the location information of theinformation terminal 103. FIG. 15 is a diagram illustrating an exampleof a notification setting screen for location information. Thenotification setting screen includes wording 1601 for explaining thesettings, an enable (“receive”) button 1602, and a disable (“do notreceive”) button 1603. When a touch operation on the enable button 1602is confirmed, the calculation circuit 209 instructs the informationprocessing apparatus 102 that notifications have been turned on orauthorized. On the other hand, when a touch operation on the disablebutton 1603 is confirmed, the calculation circuit 209 instructs theinformation processing apparatus 102 that notifications have been turnedoff or unauthorized. The calculation circuit 206 registers theinformation indicating notifications being on or off in the record ofthe advertisement viewer 107. The calculation circuit 206 informs onlythe advertisement viewer 107 (the information terminal 103) withnotifications set to on of the location information of the articleholder 108.

The information of the advertisement viewer 107 registered via thescreens described above may be changed at any time by the advertisementviewer 107 by using the information terminal 103 to log in to thewebsite of the advertisement service provided by the informationprocessing apparatus 102. Also, before the information terminal 103 ofthe advertisement viewer 107 is actually informed of the locationinformation of the article holder 108 (the information terminal 104),data of the notification setting screen may be transmitted from theinformation processing apparatus 102 to the information terminal 103.This allows for the location information of the article holder 108 to beinformed only in cases where consent (notifications turned on) of theadvertisement viewer 107 has been obtained.

Note that, in this example, the information terminal 103 uses a browserapplication and repeats the reception of the input screen and thetransmission of registration information while communicating with theinformation processing apparatus 102. However, a dedicated applicationincluding the input screens may be installed in the information terminal103, and all of the information may be transmitted as registrationinformation from the information terminal 103 to the informationprocessing apparatus 102 after all the information has finished beinginput.

Invoice Processing for Registration and Advertisement Cost ofAdvertisement Requester

Next, the operations of the information processing apparatus 102relating to the registration procedure 115 of the advertisementrequester 109 and an invoicing procedure 116 for the advertisementrequester 109 illustrated in FIG. 1 . FIG. 16 is a block diagramillustrating an information terminal 105 of the advertisement requester109 and the information processing apparatus 102 and the functionalconfiguration thereof relating to a request procedure 115 and theinvoicing procedure 116. The functional configuration of the informationprocessing apparatus 102 is as described in reference to FIG. 2 .

The information terminal 105 is a computer device such as a stationarycomputer, for example. The information terminal 105 may also be a smallcomputer device, such as a smart phone, a tablet computer, a laptopcomputer, or the like. The information terminal 105 includes atransmission circuit 1701, a calculation circuit 1702, an input circuit1703, a display device 1704, and a storage device 1705.

The calculation circuit 1702 is a CPU, for example, that loads a programstored on a ROM of the storage device 1705 on a RAM of the storagedevice 1705 and executes the program. By the program being executed, thecalculation circuit 1702 implements various operations of theinformation terminal 105, such as controlling the circuits inside theinformation terminal 105, communicating with an external device, and thelike.

The transmission circuit 1701 is a communication interface between anexternal device and the information terminal 105 and is compliant withat least one wired and/or wireless communication protocol. Typically,the transmission circuit 1701 includes a wired communication interfacecompliant with Ethernet (registered trademark) and a communicationinterface compliant with a general-purpose wireless communicationprotocol, such as wireless LAN or Bluetooth (registered trademark). Notethat in FIG. 16 , the information terminal 105 and the informationprocessing apparatus 102 are illustrated as communicating directly.However, they may communicate via a network.

The storage device 1705 includes the ROM, the RAM, an HDD, and the like.The ROM stores programs executed by the calculation circuit 1702,various setting values of the information terminal 105, and the like.The RAM is used when the calculation circuit 1702 executes programs, asbuffer memory for various data, and the like. The HDD (or a largecapacity storage device) stores information of the advertisementrequester 109, user data, applications, and the like.

The display device 1704 is a liquid crystal display, for example, anddisplays various displays according to a program executed by thecalculation circuit 1702. The display device 1704 may be a touchdisplay. Also, the display device 1704 may be an external device of theinformation terminal 105.

The input circuit 1703 is an input device, such as a touch panel, akeyboard, or a mouse, operated by the advertisement requester 109.

As described above, the calculation circuit 206 of the informationprocessing apparatus 102 causes the information processing apparatus 102to function as a web server by executing a program, for example, andprovides information for displaying various input forms to the webbrowser that accesses the web server.

The storage device 207 records the information relating to theregistration procedure 115 and the invoicing procedure 116. Also, thestorage device 207 functions as a web server and, when accessed by theinformation terminal 105, records a website for inputting and displayingthe information relating to the registration procedure 115 and theinvoicing procedure 116.

FIG. 17 is a flowchart relating to the operations of the informationprocessing apparatus 102 relating to registration (the registrationprocedure 115 of FIG. 1 ) of the advertisement requester 109.

In step S1802, the calculation circuit 206 replies to access to awebsite (URL) for advertisement requester registration from the webbrowser application run by the information terminal 105 and transmitsdata of an input screen for advertisement requester information to theinformation terminal 105. The calculation circuit 1702 uses the webbrowser application and causes the display device 1704 to display theinput screen for requester information.

FIG. 18A is a diagram schematically illustrating the appearance of theinformation terminal 105 when the requester information input screen isdisplayed.

In this example, the entire display device 1704 is used for the webbrowser application. Also, because the advertisement requester isgenerally considered to be a company and not an individual, an inputscreen for registering company information as the requester informationis illustrated. However, individual registration may also be performed.

The advertisement requester 109 operates the input circuit 1703 andinputs the requester information in each input item. Also, though notillustrated, a password for logging into the website for registrants maybe input. Furthermore, by operating a registration button 1902 viaclicking or the like, the information terminal 105 is informed of theinput of the requester information. When an operation on theregistration button 1902 is confirmed, the calculation circuit 1702transmits the items input on the screen to the information processingapparatus 102. When the requester information is received, thecalculation circuit 206 generates an advertisement requester recordincluding a unique user ID, stores the received requester information ina predetermined field in the record, and stores this in the storagedevice 207. Note that in a case where an operation of a cancel button1903 is detected, the calculation circuit 1702 confirms with the user(the advertisement requester) the stopping of registration processing.In a case where stopping of the registration processing is confirmed,the calculation circuit 1702 instructs the information processingapparatus 102 to cancel the registration processing and closes the inputscreen displayed on the browser application. In a case where stopping ofthe registration processing is cancelled, the calculation circuit 1702continues displaying the input screen. When the calculation circuit 206of the information processing apparatus 102 is instructed by theinformation terminal 105 to cancel the registration processing, theregistration processing ends.

In step S1803, the calculation circuit 206 transmits data of theinformation input screen of the target article to the informationterminal 105. FIG. 18B is a diagram illustrating an example of aninformation input screen 2001 for the target article 106. This screenmay be the same screen used by the article holder to register the targetarticle 106. However, in this example, more detailed information is ableto be registered. In order to increase the subject recognition accuracyof the information processing apparatus 102, input of an image of thetarget article may be made necessary. Note that information of thetarget article is not limited to that illustrated in the diagram. Whenan operation on a registration button 2202 is confirmed, the calculationcircuit 1702 transmits the input target article information to theinformation processing apparatus 102. When the target articleinformation is received, the calculation circuit 206 registers theinformation in the advertisement requester record in the storage device207. This ends the description of the registration processing for theadvertisement requester 109. Note that the advertisement requester 109that is already registered may change the requester information, add ordelete the target article, change in the information of the targetarticle, and the like by accessing the website for registrants providedby the information processing apparatus 102.

Note that to facilitate explanation and understanding, the descriptionthereof is omitted. However, in reality, processes, such as binding ofagreements relating the service provision, registering a fee paymentmethod, and the like may be included between the advertisement requesterand the advertisement company.

FIG. 19 is a flowchart relating to the operations of the informationprocessing apparatus 102 relating to invoicing for the advertisementrate (invoice 116 in FIG. 1 ). Note that in the present embodiment, theadvertisement requester 109 performs a bank transfer of the payment ofthe advertisement rate, and a service for viewing advertisement ratesthat require payment is provided as the invoicing processing for theadvertisement rate.

In step S2102, the calculation circuit 206 replies to access to awebsite (URL) for registered advertisement requesters from the webbrowser application run by the information terminal 105 and transmitsdata of a login screen for the advertisement requester to theinformation terminal 105. The calculation circuit 1702 uses the webbrowser application and causes the display device 1704 to display thelogin screen.

FIG. 20A is a diagram schematically illustrating the appearance of theinformation terminal 105 when a login screen 2201 for the advertisementrequester is displayed.

In this example, login is received via a combination of the requester'sname and password. However, any known method may be used, such as usinga combination of a user ID and password, using biometric information,and the like.

The advertisement requester 109 inputs the requester name and passwordby operating the input circuit 1703. Furthermore, by operating a loginbutton 2202 via clicking or the like, the information terminal 105 isinformed of the input of the login information. When an operation on thelogin button 2202 is confirmed, the calculation circuit 1702 transmitsthe items input on the screen to the information processing apparatus102. When the login information is received, the calculation circuit 206searches the advertisement requester record and determines whether ornot the login information is correct. In a case where the logininformation is determined to be correct, the calculation circuit 206transmits, to the information terminal 105, the data of a service menuscreen for registered advertisement requesters. The calculation circuit1702 causes the display device 1704 to display the service menu screen.

Here, the advertisement requester 109 selects the invoice amount viewfrom the service menu screen. The calculation circuit 1702 transmits arequest to view the invoice amount corresponding to the selection to theinformation processing apparatus 102. When the request to view theinvoice amount is received, the calculation circuit 206 references theadvertisement requester record, obtains the billing information for apredetermined cumulative time period, and transmits this to theinformation terminal 105.

Note that the advertisement rate may be calculated based on the numberof times the advertisement (the location information of the articleholder 108) was provided to advertisement viewers for the target articleregistered by the advertisement requester 109, for example. Note that adetailed method of calculating the advertisement rate may be changedaccording to an agreement. Such plausible examples include a chargesystem in which a fixed amount is paid or in which the advertisementrate for one advertisement is reduced depending the more the number oftimes the advertisement is provided is increased, and the like. Also,the charge for one advertisement may be different depending on the costor side of the article. The relationships between the number of timesand the charge illustrated in FIG. 20B are simply examples. As describedabove, in the tracking processing, the calculation circuit 206 measuresthe number of times the advertisement viewer 107 is informed withlocation information for each target article 106 and stores this.Accordingly, the calculation circuit 206 may generate billinginformation containing the totaled advertisement rate for theadvertisement requester 109 corresponding to the target article 106.

FIG. 20B is a diagram schematically illustrating the appearance of theinformation terminal 105 when a display screen 2301 of the billinginformation is displayed. The advertisement requester 109 can confirmthe details and the amount of the invoiced advertisement rate via thedisplay screen 2301. When an operation of a logout button 2302 isconfirmed, the calculation circuit 1702 transmits a logout request tothe information processing apparatus 102. When the logout request isreceived, the calculation circuit 206 executes logout processing.Thereafter, the advertisement requester 109 completes payment of anadvertisement rate 118 by transferring the designated amount to the bankaccount of the advertisement company 100.

According to the advertisement system of the present embodiment, by theadvertisement viewer registering an article they are interested in asthe target article, the advertisement viewer can be informed that theactual target article is nearby. Accordingly, the actual target articlecan be confirmed by taking advantage of the fact that the target articleis nearby. This allows for information that is unable to be obtainedfrom the state of the article when on display in an advertisement or inthe store to be easily obtained. Also, in a case where the articleholder is carrying or using the target article, trends relating to theage, gender, and the like of the users can be known, allowing for aninformed choice to be made when purchasing the target article.

Also, by registering the article they own, the article holder can obtaina consideration while going about their everyday life without giving itmuch thought. Also, providing the location information at undesiredtimes can be avoided by temporarily setting the location information tobe not provided.

Furthermore, by registering an article they wish to advertise, theadvertisement requester can advertise using actual owners toadvertisement viewer that are interested in the article. This isadvantageous in that a more effective advertisement effect can beachieved.

Second Embodiment

Next, the second embodiment of the present disclosure will be described.In the first embodiment, the image capture apparatus 101 is placed at afixed location outside, and the information terminal 103 of theadvertisement viewer 107 that moves is informed of the locationinformation of the target article 106. The present embodiment isdirected at an environment in which the image capture apparatus alsomoves.

FIGS. 21A to 21B is a schematic diagram of the inside of a passengervehicle that includes both an image capture apparatus of anadvertisement system and an information terminal of an advertisementviewer. In the examples illustrated in FIGS. 21A and 21B, the locationof the image capture apparatus and the information terminal are arrangeddifferently. In this example, the passenger vehicle is capable ofautomated driving, but the passenger vehicle may not be capable ofautomated driving. The advertisement system of the present embodimentmay be executed using the information processing apparatus 102 and theinformation terminals 103 to 105 described in reference to the firstembodiment. Thus, the descriptions of the configurations of the devicesare omitted.

An image capture apparatus 2401 captures an image of the surroundingsoutside the vehicle including the left and right sides of the vehicleand transmits the video data to the information processing apparatus 102of the advertisement system. As illustrated in FIG. 21B, the imagecapture apparatus 2401 may include a plurality of image captureapparatuses. An image capture apparatus 2402 is provided to capture animage of a person riding in the vehicle. The information processingapparatus 102 may determine whether or not a registered advertisementviewer is in the vehicle from the video captured by the image captureapparatus 2402.

A display device 2403 is a transparent display device doubling as a sidewindow in the example of FIG. 21A and a display device provided on aseat back in the example of FIG. 21B. The display device 2403corresponds to the display device 210 of the information terminal 103 ofthe advertisement viewer 107 in the first embodiment. FIGS. 21A and 21Billustrate examples in which the message 401 is displayed as targetarticle information in a case where a vehicle registered as the targetarticle 106 by the advertisement viewer 107 riding in the vehicle islocated nearby.

In the example of FIG. 21A, by a frame-like indicator being displayedaround a region of the target article 106 on the display device 2403,which is a transparent display device, an augmented reality (AR) displayis displayed in which the frame-like indicator is superimposed on whatis seen through the display device 2403. For example, even in a casewhere the advertisement viewer 107 is working inside the vehicle, thenumber of times when the actual target article 106 can be seen can beincreased.

Also, FIG. 21B is a diagram illustrating an example display of thetarget article information on the display device 2403, which is adisplay device for the rear portion of a seat. In this example, themessage 401 indicating whether the target article is on the left side orthe right side of the vehicle is displayed as the target articleinformation. Note that video of the target article captured by the imagecapture apparatus 2401 may also be displayed together with theinformation. Also, the target article information may be displayed onthe display device 2403 on the left side seat only in a case where thetarget article 106 is on the left side. The same applies to the displaydevice 2403 on the right-side seat. Also, as illustrated in FIG. 21B,the target article information may be displayed only on the displaydevice 2403 of the seat where the advertisement viewer 107 is sitting asdetermined by the image capture apparatus 2402. The above-describedspecific display device control methods for the display device 2403 thatdisplays the target article information may be combined or no suchcontrol may be executed.

The display device for the rear portion of the seat is a touch display,and when a touch operation of the end button 402 is detected, thecalculation circuit 209 ends the display of the message 401.

FIG. 22 is a block diagram illustrating the information processingapparatus 102 of the advertisement system of the present embodiment anda configuration example of the components provided in the vehicle. Theconfiguration of the information processing apparatus 102 is the same asin the first embodiment. Also, the image capture apparatuses 2401, 2402and the information terminal 103 are provided in a vehicle 2601. Theconfigurations of the image capture apparatuses 2401, 2402 are the sameas the image capture apparatus 101 of the first embodiment. Also theconfiguration of the information terminal 103 is the same as in thefirst embodiment. Note that in the example illustrated in FIG. 21B, theinformation terminal 103 is provided with two display devices capable ofindependent display. The information terminal 104 of the article holder108 and the information terminal 105 of the advertisement requester 109are not illustrated, but are the same as in the first embodiment.

The operations of the information processing apparatus 102 in theadvertisement system of the present embodiment are basically the same asthe operations in the first embodiment described with reference to FIGS.3A to 3C, and thus descriptions thereof will be omitted. However, in acase where the image capture apparatus 101 moves as in the presentembodiment, in recognition processing, the article holder 108 isdifficult to recognize from the video data. Thus, in a case where thetarget article information is a vehicle, the vehicle registration number(the number on the number plate) is registered as the target articleinformation, and in step S307, the calculation circuit 206 determineswhether the target article 106 is present on the basis of number platerecognition. Note that the vehicle registration number used inregistration and recognition may be only the designated number (thefour-digit number portion ranging from xxx1 to 99-99).

Note that in a case where the vehicle is capable of automated driving ofSAE J3016 level 2 or greater, the vehicle and the automated drivingsystem may be connected in a manner such that the target article 106 istracked and the vehicle is automatically driven to the location (or alocation nearby) where the target article 106 is. For example, in a casewhere the target article 106 (vehicle) registered by the advertisementviewer 107 is found in a carpark, the vehicle is automatically parked inan open parking space next to or nearby the parking space of the targetarticle 106. In this manner, if the advertisement viewer 107 is in amoving body, the target article 106 can be viewed at a close proximity.

The present embodiment can obtain the same effects as that of the firstembodiment. Also, information of the target article 106 can be obtainedin a timely manner even while moving in a vehicle.

Other Embodiments

In the embodiments described above, the advertisement viewer accesses asite of the advertisement system and explicitly registers in advance anarticle they are interested in. However, the advertisement viewer mayregister an article they are interested in by another method. Forexample, an article that the advertisement viewer may have interest inmay be identified and automatically registered on the basis of thebrowsing history of the web browser application run on the informationterminal of the advertisement viewer. In this case, consent to use thebrowsing history is obtained in the registration processing of theadvertisement viewer. Technology for identifying an article a viewer maybe interested in from browsing history include targeting advertisementtechnology using cookies of the web browser. Also, in a case where theadvertisement viewer is shown in the video data and the direction theadvertisement viewer is looking in (face orientation) can be detected,an article the advertisement viewer is looked at may be registered as anarticle they are interested in. For example, an article may beidentified by the calculation circuit 206 recognizing an article in thedirection the face is facing.

Embodiment(s) of the present disclosure can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present disclosure includes exemplary embodiments, it is to beunderstood that the disclosure is not limited to the disclosed exemplaryembodiments. The scope of the following claims is to be accorded thebroadest interpretation so as to encompass all such modifications andequivalent structures and functions.

What is claimed is:
 1. An information processing apparatus comprisingone or more processors that execute a program stored in a memory andfunction as: a detection unit configured to analyze video data obtainedfrom an image capture apparatus to detect an article that has beenpre-registered; a search unit configured to, in a case where an articlethat is determined to be a property of an article holder that has beenpre-registered is detected, search for an advertisement viewer locatedwithin a range of a predetermined distance from the article holder fromamong advertisement viewers that have registered the article as anarticle they are interested in; and an informing unit configured toinform an advertisement viewer found by the search unit of locationinformation of the article.